This invention relates to an information input/output vice. More particularly, it relates to an information put/output device for superpositionally displaying an formation displayed on an image plane on another image plane which any other information is being displayed. As will be clarified subsequently, the present invention relates more specifically to an information input/output device in which any other information of plural image planes are presented to a user simultaneously in physically discrete relationship in the vertical direction to contribute to improving the efficiency in the information processing operation.
There has so far been known a method for arraying the information in the same plane as an environment in which the information processing operation is performed as variable different information is displayed on an image plane of a display apparatus. This is generally called graphical user interface (GUI). In GUI, there is usually demonstrated on the display image plane (screen) of a display device one or more windows in which to demonstration the information. In this case, the window may be deemed to be a unit of representation for presenting and supervising the information.
Of the plural windows demonstrated on a screen of a display device, in a routine window displaying system, only the window on the foremost foreground has its entire area displayed, whilst the windows hidden behind this window is partially displayed or hidden completely.
Thus, for accessing the hidden window, the window to be demonstrated is activated and moved to the foremost foreground on the screen.
As a result, the window, so far activated, is rendered inactive, by way of interchanging the foreground/background relationship as far as display is concerned.
For demonstrating plural windows on one screen, a user interface is required for quickly specifying or interchanging the respective windows. For example, in Windows (registered trademark of an OS developed by MICROSOFT INC.), switching to the windows to be activated is performed quickly by arranging a window-switching shortcut icon on a task bar or on a desktop or by allocating a window-switching shortcut key to the keyboard.
As another method for presenting plural information on an screen, there is known a method for switchingly demonstrating images. There is also known a method of increasing the volume of the information that can be demonstrated by increasing the image size of the display device and enhancing the resolution or by providing plural display devices to increase the display images.
Also, in a number of application softwares, employing a touch panel display, including, in the first place, an application software employing a touch panel display, a flat GUI is mainly used. As a pointing operation, a button image demonstrated on an image is pressed with a finger on a touch panel, or a mouse button is clicked. Responsive to this actuation, the button image is high-lighted, changed in color, caused to blink or changed to an yielded bit map image.
Meanwhile, among conventional display devices for superposed display of plural images, there are known such a display device employing e.g., a half mirror, and such a display device including an optical device having a similar function, called a flat reflection spectroscope, a combiner or a beam splitter. These optical devices are herein collectively termed a half mirror.
In this type of the conventional devices, there is disclosed in JP Patent Kokai JP-10-123974 a switching display device having a half mirror and configured for displaying two or more image information. The half mirror is adapted for transmitting or reflecting the light radiated from two planar light sources and which has passed through respective transparent tablet plates. There is also disclosed in JP Patent Kokai JP-7-287186 a goggle type display device having two liquid crystal display devices and a half mirror. There is disclosed in JP Patent Kokai JP-10-79906 an arrangement of a liquid crystal television monitor having two liquid crystal display panels, a half mirror for synthesizing two images, a circuit for sampling an image demonstrated on the two liquid crystal display panels and a signal switching switch for supplying sampled image signals to a liquid crystal display panel driver. Moreover, there is disclosed in JP Patent Kokai JP-9-507311 an arrangement of an image synthesizing device for generating a synthesis of plural images/visible phenomena, in which there is provided, from the viewpoint of a viewer, at least one projected real image demonstrated in a neighboring style in a fore-and-aft relation, adjacent to one another, in superposition or in a intersecting relation.
In addition, there is proposed, in JP Patent Kokai JP-04-077087, a stereoscopic image display device in which plural images associated with respective distances form an imaging point to different objects are demonstrated in apparent superposition at plural display positions having respective different optical viewing distances relative to the viewer so that the magnitudes of said optical viewing distances will appear to be the magnitudes of the aforementioned distances, in which the necessity of using special goggles, display units or movable parts of the optical system will be eliminated by arranging plural transparent display units demonstrating plural images at plural display positions.
The display routinely used for information display is a flat type display, such as CRT or liquid crystal display panel, adapted for displaying plural sorts of the information on a sole plane (image plane, e.g., display screen plane).
In the conventional GUI employing this sort of the display, a window, which may be said to be a virtual paper sheet, as an information management unit, and the text, a bit map image, an operating icon or a menu is arranged on the window.
As an area in which to demonstrate plural windows, there is provided an area termed a xe2x80x9cdesk top areaxe2x80x9d which may be said to be a virtual desk surface. On this desk top area the respective windows are arrayed in superposition or arranged as icons to permit a user to prosecute his or her operations.
However, the conventional information display device and the information input/output devices suffer from the following problems.
The first problem is that, if many sorts of the information are displayed on a conventional GUI image plane, it is not easy for the operator to grasp immediately to which point his or her attention should be directed.
The reason is that, in the conventional display device, there is no alternative but to arrange the information on the sole screen plane (image plane) and, if desired to increase the number of the information items to be demonstrated simultaneously, there is no alternative but to reduce the display size of the individual information or to display the information in superposition, as a result of which the information becomes difficult to view for the operator.
If the information to which the attention of the user is to be directed is highlighted, or surrounded by a frame to emphasize the information, the emphasized information is insufficient in definiteness in representation, since the emphasized information is still arrayed on the same plane as the other information.
The second problem is that searching a window hidden behind a foreground window and switching a background window to the foreground represents a cumbersome operation.
In a routine window displaying system, there are occasions where the window on the foreground is displayed in its entirety, whilst the remaining windows are hidden partially or in their entirety by other windows. If the windows are hidden in this manner, a targeted window is difficult to search and, in addition, the information on the other windows are difficult to refer to while the windows or the information represented on the windows cannot be viewed at a glance.
For reliably demonstrating the hidden windows, the window(s) in question need to be activated, however, if plural windows are demonstrated on the same image plane, it is difficult to designate a targeted window directly on the screen.
On the other hand, there are occasions where an image field of a window so far in the active state is rendered inactive, as a result of switching another window into the active state, so that the display on the foreground is changed over to that on the background and vice versa, with a result that the information so far viewed by the user now is hidden behind the active window and hence becomes invisible.
For solving this problem, there is used a method which makes use simultaneously of a user interface for quickly designating or changing over the respective windows. Taking the Windows (registered trademark) as an example, it is retained to be possible to carry out the switching of the active windows quickly by such methods as arranging a window switching shortcut in a general-purpose area such as a task bar, or allocating a window switching shortcut key to a keyboard.
However, these conventional methods are not up to the requirements to demonstrate the plural windows simultaneously.
For demonstrating plural windows simultaneously, the window size may be adjusted to array the windows side-by-side on a sole display screen, one or more displays are added to provide display screens, or the image size may be increased.
However, if the plural windows are arranged side-by-side on a sole image screen, the volume of the information that can be displayed per window is decreased.
If plural display devices are added to add image screens or the image size is increased, the distance (stroke) along which the viewing line of the user in operation is prolonged, such that, in the case of an operation in which respective windows need to be referenced frequently, the load imposed on the operator is increased.
The third problem resides in that, in the display screen switching method, the movement distance (stroke) of the viewing line for the user is the same as before, however, since the user cannot perform an operation o n the display screen as switched, as the he or she refers to the information displayed on the display screen previous to switching, the volume of the information that needs to be memorized by the operator is increased.
The fourth problem resides in that operational feedback which affords the operating feeling to the user cannot be furnished by GUI.
In routine GUI, such as a computer or a communication terminal, a button image arranged on the display screen is planar and, if a user presses the button image, the bit map image is highlighted, changed in color blinking or changed to a recessed bit map image, by way of a feedback operation to the button image pressing operation.
However, these expressions are poor in real feeling as the reply or response to the pressing-down of the button image, such that the user cannot readily grasp or sense whether or not the system has already responded to the operation by the user and hence the user may feel uneasy whether or not his or her operation has been a correct one.
In particular, if, in case of an application software employing a touch panel display, a button image from a touch panel is pressed the pressing-down feeling is insufficient, so that the user is unable to comprehend readily that the system has reacted to the pressing down operation.
The result is that the user cannot confirm readily that his or her previous operation has been a correct one, such that he or she has to press down the touch display with a needlessly large force or an unnecessary number of times.
The fifth problem resides in that it is not presupposed in the conventional superpositional display device that an superposing image plane and an image plane to be superposed present respective different information, the image switching is a mere on/off control, whilst there lacks in the conventional superpositional display device the function of dynamically controlling the brightness and/or contrast of the image required when effectuating stepped switching or superposition of the respective image planes.
In particular, on the side of the image plane of an image to be overlain, overlaid with an overlying portion, the portion to be displayed in superposition is masked to render the portion to be visible more readily. However, this masking processing cannot be performed dynamically to follow up with changes in the superposing information.
It is therefore an object of the present invention to provide an information displaying device whereby an information displayed heretofore on a sole display screen (or image plane) is more comprehensible and the attention of the user may be guided more spontaneously to the crucial information.
It is another object of the present invention to provide a display device in which, if many pieces of information are displayed on a sole display field, which is the crucial information or on which portion focussing is to be applied can be clearly understood, and in which, even if the information and/or the windows are superposed with one another, underlying information can be viewed to a certain extent.
It is still another object of the present invention to provide an information input/output device in which an operating environment with higher operating feeling can be realized for a routine GUI such as a computer or a communication terminal to say nothing of the application software employing a touch panel display.
It is yet another object of the present invention to provide an information input/output device wherein effective superpositional display can be realized between an image plane of an superposing image and an image plane of an image to be superposed, as well as adjustment of brightness or contrast of both the image planes or masking control for the portion to be displayed in superposition can be performed dynamically. Other objects of the present invention will become readily apparent from the following description and the claims.
According to a first aspect of the present invention, there is provided a super positional image display type information input/output device including (a) an image generator unit generating (and outputting) an image (first image) for display in superposition and an image (second image) to be superposed from a given image, and (b) superpositional image display unit providing a (first) display image plane of the image for display in superposition and an image plane of an image to be superposed in separated levels from each other in a superposed state in a manner that the (first) image for display in superposition will appear to be floated on or recessed from a display image plane of the (second) image to be superposed. The superpositional image display type information input/output device according to the present invention further includes a mask pattern generating module generating a mask pattern for the (first) image for display in superposition, and an image generating module generating the (second) image to be superposed from the mask pattern and the given image.
In more detail, according to a second aspect of the present invention there is provided an superpositional image display type information input/output device comprising the following elements (a) to (e)
(a) an image generator unit that extracts a portion to be displayed in superposition from a given image to output an extracted image portion as an superposing image, and that generates (and outputs) a mask pattern for the superposing image;
(b) a first effector unit that receives the superposing image output by the image generator unit, processes the received superposing image with designated image processing in accordance with effect designating information, to output the processed superposing image;
(c) a second effector unit that receives the mask pattern output by the image generator unit, processes the received mask pattern with designated image processing in accordance with the effect designating information, to output the processed mask pattern image,
(d) a base image generating unit that synthesizes the mask pattern image output by the second effector unit and the given image to generate and output a base image; and
(e) a superpositional image display unit that superpositionally displays the processed superposing image and the base image as demonstrated on first and second display image planes.
According to a third aspect of the present invention, the image generator unit comprises the following elements:
an superposing image extracting module configured for extracting an image portion to be displayed in superposition from an image to be displayed in superposition, in accordance with an superposing area designating information for designating an superposing area on an image, to output an extracted image as an superposing image; and
a mask pattern generating module configured for generating a mask pattern based on the superposing area designating information.
Moreover, according to a fourth aspect of the present invention, the image generator unit comprises the following elements:
an superposing information analyzing module configured for generating the superposing image generating information necessary for generating an superposing image and the superposing area designating information in accordance with the superpositional position designating information given as information other than an image;
an superposing image generating module configured for generating and outputting an superposing image in accordance with the superposing image generating information; and
a mask pattern generating module configured for generating a mask pattern based on the superposing image generating information.
According to a fifth aspect of the present invention, the image generator unit may comprise:
an superposing image analysis module receiving the superposing information as an superposing image to analyze the shape of the superposing image and an superposing position to generate (and output) the superposing area designating information in accordance with the resulting superposing position designating information, and
a mask pattern generating module generating a mask pattern based on the superposing area designating information.
According to a sixth aspect of the present invention, the superposing information is given as an superposing image with a size or an image format which is the same as the size or the format of a given reference image, an superposing display position being pre-set, the superposing image being directly fed to the first effector module. The image generator unit includes a mask pattern generating unit generating a mask pattern based on the superposing area designating information.
According to a seventh aspect of the present invention, there may be provided a brightness/contrast controller configured for adjusting the brightness and/or the contrast of the image for display in superposition and the base image depending on characteristics of the displays, to supply the resultant images to respective displays.
According to a further aspect of the present invention, there may also be provided a switching module configured for switching over control of displays for the image for display in superposition output by the first effector module and the base age output by the base image generating module.
According to the present invention, there may also be provided a switching-over module configured for switching-over between the image for display in superposition output by the first effector module and the base image output by the base image generating module under adjustment of the brightness or contrast of the images.